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HIGH MOMENT PARTIAL SUM PROCESSES OF RESIDUALS IN 
GARCH MODELS AND THEIR APPLICATIONS 1 

By Reg Kulperger and Hao Yu 
University of Western Ontario 

In this paper we construct high moment partial sum processes 
based on residuals of a GARCH model when the mean is known to 
be 0. We consider partial sums of fcth powers of residuals, CUSUM 
processes and self-normalized partial sum processes. The fcth power 
partial sum process converges to a Brownian process plus a correc- 
tion term, where the correction term depends on the fcth moment 
fj,k °f the innovation sequence. If fj,k = 0, then the correction term 
is and, thus, the fcth power partial sum process converges weakly 
to the same Gaussian process as does the fcth power partial sum of 
the i.i.d. innovations sequence. In particular, since /ii = 0, this holds 
for the first moment partial sum process, but fails for the second 
moment partial sum process. We also consider the CUSUM and the 
self-normalized processes, that is, standardized by the residual sam- 
ple variance. These behave as if the residuals were asymptotically 
i.i.d. We also study the joint distribution of the fcth and (fc + l)st 
self-normalized partial sum processes. Applications to change-point 
problems and goodness-of-fit are considered, in particular, CUSUM 
statistics for testing GARCH model structure change and the Jarque- 
Bera omnibus statistic for testing normality of the unobservable in- 
novation distribution of a GARCH model. The use of residuals for 
constructing a kernel density function estimation of the innovation 
distribution is discussed. 

1. Introduction and results. In nonlinear time series and in particu- 
lar econometric and discrete time financial modeling, Engle's [12] ARCH 
model plays a fundamental role; see [10], or the volume edited by Rossi 
[20]. The ARCH model has been generalized to GARCH by Bollerslev [5]. 
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and 



(1.2) 



t-p 



i=l j=l 



where 



(1.3) a > 0, at > 0, 1 < i < p, > 0, 1 < j < q 

are constants, and the innovations process {ej, — oo < t < oo} is a sequence 
of i.i.d. random variables (r.v.'s). When eq has a finite kth. moment, denote 
/life = E(eq). A usual GARCH model assumption also is: 



Throughout this paper we assume that (1.1)-(1.4) hold, so that, by defini- 
tion, fi\ = and fi2 = 1. 

The existence of a unique strictly stationary solution of (1.1) and (1.2) 
is well established. See [6, 7] for details. In this paper a minimal set of 
conditions in [6, 7] for the existence and stationarity of the GARCH(p, q) 
sequence {Xt, — oo < t < oo} is assumed, plus the assumption (1.4). 

Estimation of the parameter 9 = (oto, a\ , . . . , a p , Pi , . . . , P g ) has been inves- 
tigated by several authors. We only cite those relevant to our investigation. 
Throughout this paper we assume that n is an estimator of 6 based on a 
sample Xq,X\, . . . ,X n , and that it is y/n consistent, that is, 



where we use | • | to denote the maximum norm of vectors or matrices. Re- 
cently Berkes, Horvath and Kokoszka [3] studied the asymptotic properties 
of the quasi-maximum likelihood estimator for 6 in GARCH(p, q) models 
under mild conditions. Berkes and Horvath [1] have shown that the quasi- 
maximum likelihood estimator cannot be \/n consistent if £?|£o| = 00 f° r 
some < k < 4. Hall and Yao [14] also studied inference for GARCH mod- 
els under slightly stronger assumptions on the parameters. To remove such 
limitations as the finite fourth innovations moment, Berkes and Horvath [1] 
have used an arbitrary density function to replace the normal function used 
in the quasi-maximum likelihood and obtain the asymptotic properties (con- 
sistency and normality) under mild conditions. In particular, they show that 
the quasi-maximum likelihood estimator based on the standard symmetric 
exponential density function is y/n consistent only if Esq < oo. 



The innovations process is a sequence 



(1.4) 



of i.i.d. mean and variance 1 r.v.'s. 



(1.5) 



yfr\e n -o\ = Op{\) 
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The main goal of this paper is to construct high moment partial sum 
processes of residuals in a GARCH model. Since GARCH processes are 
defined in terms of their conditional variances, it is natural to construct 
model diagnostics in terms of sums or sums of squares of either the ob- 
served raw data or the driving noise estimates (residuals). As is well known 
from regression and linear time series, diagnostics based on residuals are 
often better tools. Sample skewness and kurtosis are based on sums of third 
and fourth powers, respectively, and thus, are also interesting diagnostic 
tools. 

Usually the conditional variance of of (1.2) is estimated by 

p q 

of = d + ®i x t-i + h°t-j > R = max(p, q)<t<n. 

i=l j=l 

One problem with the above estimation is that the initial conditional vari- 
ance estimates o"^_ l5 o\_ 2 ^ ■ ■ ■ , <j\_ q must be given. Hall and Yao [14] give 
an infinite-order moving average representation for of under the condition 
that Yh=i a i + Sj=i /5j < 1- A recursive representation of of is obtained by 
Berkes, Horvath and Kokoszka [3] under the weaker condition (than [14]) 
that Y2j=i/3j < 1 5 which we are already assuming for the stationarity of 
{X t , —oo < t < oo} under the conditions of Bougerol and Picard [6, 7]. We 
use these later results of Berkes, Horvath and Kokoszka [3] to construct of , 
adapt their notation and conditions and give them here in some detail. 

Let u= (s,t) G s = (s , si, . . . , s p ) G W +1 and t = (ti,...,t ff ) G 

M. q . Define Cj(u), i = 1, 2, . . . , R = m&x{q,p} by: if q > p, then 

C (U) = S /( 1 + +tq)), 

ci(u) = si, 

c 2 (u) = s 2 -Mici(u), 

Cp(u) = Sp + tic p _i(u) H htp-ici(u), 

Cp+i = t 1 c p (u) H htpCi(u), 

Cg(u) = tlCg_l(u) H h tq_lCl(u), 

and if q < p, the equations above are replaced with 

c (u) = s /( 1 -(*! + ■•■ + 
ci(u) = Si, 
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c 2 (u) = s 2 + hc 1 (u), 



c g+ i(u) = s q+1 +tic g (u) H htgCi(u), 

c p (u) = Sp + iiCp_i(u) H h tgC p _g(u). 

For i> R, define 

Ci(ll) = tlQ_l(ll) + t 2 Q-2(u) H h ^Cj-^tl). 

Let < u < u, 0<po<l 5 Q1L< Po, and define the parameter space as 

= {u : t\ + ■ ■ ■ + t q < po,u< min(s, t) < max(s, t) < u}. 

In the rest of this paper we replace (1.3) with the stronger condition 

(1.6) 9 is in the interior of 0. 

Now we are ready to give the recursive representation of the conditional 
variances by previous observations as given by Berkes, Horvath and Kokoszka 
[3]. Define 



2 

•i" 
i=l 



Then erf (u) exists with probability one for all u G 0. Also, a 2 in (1.2) has 
the representation 



(i.7) a 2 t =a 2 t {e) = c Q {e)+Y J c l {e)xl l . 

Given n , we can estimate a 2 (0) by 



r 2/ 



(1.8) a 2 = a 2 {6 n ) = c (6 n ) + £ c*(0 n )X 



2 

t-i- 

i=l 



In practice, we observe only Xq,X\, . . . , X n . Hence, we use a truncated form 
and define, for 1 < t < n, 

t 

(1.9) d 2 = cv(e n ) +Y J c t {e n )x 2 _ i . 

i=l 

Thus, the residual at time t is 

(1.10) e t = - 1 , l<t<n. 
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The kth {k = 1, 2, 3,4,.. .) order moment partial sum process of residuals is 
defined as 

[nu] 

(1.11) ^ fc) («)=E& 0<n<l. 

t=i 

Its counterpart based on the i.i.d. innovations is defined as 

[nu] 

(1.12) Si fc V)=]>> fc , 0<n<l. 

t=l 

Collectively in k, we refer to these as high moment partial sum processes. 
Denote 

_ / <9q(u) faf(ll) flCj(u) 9Ci(u) gCj(u) \ 

A ' \ 8a ' 9ai da p ' dft 80 q ) 

dlM _ **M _ aco(u) + Eg 1 W,l, RP+5+I 

oj(u) c*(u) + E£ 1 c i (u).X?_ i 

and 

i,{0) = E{d\ogal{0)), 

where d(-) is used as a shorthand for d(-)/du for convenience of writing. 
We need two more regularity conditions in order to state our results: 

(1.13) eo is a nondegerate random variable 
and 



(1.14) limx" c P{|e | < x} = for some ( > 0. 

x— >0 
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Lemma 3.1, (1.6), (1.14) and -E7|£o| < 00 f° r some <5 > imply the existence 
of ip(0). 

Theorem 1.1. Suppose (1.5), (1.6) and (1.14) hold, and let k > 1 ae an 
integer. If E\eo\ k < oo, i/ien 



sup 

0<m<1 



]=(S^(U) - S^( U )) + ^f%(0), y^n " 0)) 



op(1), 



where (x, y) zs i/te inner product of the vectors x and y. 



Remark 1.1. Theorem 1.1 shows that the asymptotic properties of the 
high moment partial sum process {S n (u),0 < u < 1} depend on the pa- 
rameters of the model unless = 0, which can only happen if k is an odd 
integer, i.e., not if k is even. Recall that [i\ = by (1.4). Thus, the ordinary 

partial sum process {S n (u),0 < tt < 1} behaves as though the residuals 
< i < n} were asymptotically the same as the unobservable innova- 
tions {et, 1 < t < n}. 
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By Theorem 1.1, we immediately obtain the following CUSUM result, 
Theorem 1.2. It implies that the CUSUM normalized high moment partial 
sum process {Sn (u) — uSn (1),0 < u < 1} behaves as though the residuals 
1 < t < n} were asymptotically the same as the unobservable innovations 
{e t ,l<t<n}. 

Theorem 1.2. Let k>l be an integer and suppose that (1.5), (1.6) and 
(1.14) hold. If E\e \ h < oo, i/ten 



sup 

0<«<1 V n 



^|($*>(u) -«$*>(!)) - (S^(u) -uS^(l))\ = o P (l). 



Let i^i = -E(£q — Mfc) 2 < 00 ■ Then the invariance principle for partial sums 
for an i.i.d. sequence {e^} (see, e.g., [4]) implies that 



converges weakly in the Skorokhod space D[0, 1] with J\ topology to a Brow- 
nian bridge {Bq(u), <u< 1}. Note that any topology of weak convergence 
that yields the invariance principle above could have been used here, but for 
definiteness we state that the J\ topology is used. 

The next result follows immediately from either Theorem 1.1 or Theo- 
rem 1.2. 

Corollary 1.1. Suppose (1.5), (1.6), (1.13) and (1.14) hold. If E\e \ 2k < 
oo for some integer k>\, then 

converges weakly in the Skorokhod space D[0, 1] with J\ topology to a Brow- 
nian bridge {Bo(u), < u < 1}. 

Remark 1.2. To use Corollary 1 . 1 for CUSUM tests of structural change 
of GARCH models, one needs to estimate v^. The details are left to the next 
section. 

Theorem 1.3. Suppose (1.5), (1.6) and (1.14) hold. If E\e \ h < oo for 
an integer k>l, then 



)=T,\e k t-e h t \-U k (^(O n -e)) 
n t=i z 



op(l), 



where 

^ fc (u)=£?|(egSlog<7g(fl),u)| J u e W+i+ l . 
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Remark 1.3. The above theorem shows that the sum of the absolute 
deviations \i k — e k \ depends on the parameters of the model. The existence 
of ipk( u ) follows from 

E\(4dlogo$(0),u)\<\u\E\e \ k E\dlogo%(0)\<oo 

since £o an d d log a 2 , (6) are independent and E\dlogo-Q(0)\ < oo by Lemma 
3.1. It is easy to check that ipk( u ) is symmetric about and is Lipschitz by 

|V fc (u) - Vfe(u*)| < |u- u*\E\e \ k E\dloga 2 (6)\ Vu, u* G 

where u* = (s*,t*), s* = (sg, sf , . . . , s*) G W p+1 and t* = (t\, . . . , t*) G W. 

Before formulating the next result, we need to modify the high moment 
partial sum processes of (1.11) and (1.12). The fcth-order moment residual 
centered partial sum process is defined as 

[mi] 

(1.15) fW(u)=J2&-i) k , 0<u<l, 

t=i 

where i is the sample mean of the residuals. Its counterpart based on the 
i.i.d. innovations is 

[im] 

f ( U ) = ^(e t -Ef, 0<u<l, 
t=i 

where e is the sample mean of innovations. 

Obviously, 5"? v = Ti 2 \l)/n is the sample moment estimator of fi2- In fact, 
by (3.10) a 2 n ^ — > fi2 in probability under the minimal condition ^2 < 00 • 
Since \i% = 1, then a 2 n ^ may seem to be an unnecessary estimator. However, 

it will play an important role when it is used to self- normalize Tn k \u). 
Denote er? x = Tn (l)/n, and note that it is the sample variance of the true 
innovations, except with divisor n instead of n — 1. 

Theorem 1.4. Suppose (1.5), (1.6), (1.13) and (1.14) hold and k>l 
is an integer. If £'|eo| max ^ fc ' 2 ^ < oo, then 



1 

sup —= 

0<m<1 V n 



t\u) T r { k \u) 



Op(l). 



Remark 1.4. Theorem 1.4 implies that the self- normalized (or esti- 
mated scale normalized) high moment centered partial sum process {T^ (u) / 
o^jO < u < 1} behaves as though the residuals {it, 1 < t < n} were asymp- 
totically the same as the unobservable innovations {et, 1 <t <n}. 
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Remark 1.5. When k = 1, it is easy to verify that 

sup |fW(ti)-(^W(ti)-t^W(i))|<|i|. 

0<u<l 

Hence, given Eeq < oo, Theorem 1.1 (cf. Remark 1.1) implies e = Op 
and Corollary 1.1 implies that 

^.0<.<! 

(T( n )y/n 

converges weakly in the Skorokhod space D[0, 1] to a Brownian bridge {Bq(u) 
0<u<1}. 

Let Afc = fi k /n 2 ^ 2 f° r k>l and define Ao = 1. For each k > 1, let {B( k \u), 
< u < 1} be a zero mean Gaussian process with covariance 

EB^(u)B^(v) = (X 2k -X 2 k )(uAv) 

(1.16) + k\ k _i(k\ k _i + kX k X 3 - 2X k+1 )uv 

+ fcA fc ((l - k/4)X k + kX k X 4 /4: - X k+2 )uv 

for any < u, v < 1, where u Av = min(u, v). 
If ^2k < oo, then Lemma 3.8 implies 

' -nuX k },0<u< 1 



converges weakly to the Gaussian process {B^{u),0 < u < 1}. By Theo- 
rem 1.4, we immediately obtain the following corollary. 

Corollary 1.2. 7/ (1.5), (1.6), (1.13) and (1.14) ftoJd, i/jen £|e | 2fc < 
oo /or some integer k>\ implies that 

1 ffi k \u) 



nuX k , < u < 1 

converges weakly to the Gaussian process {B( k \u),0<u< 1}. 

Remark 1.6. If the innovation distribution is symmetric about 0, then 
the covariance in (1.16) can be simplified. If k is odd, then 

EBW(u)bM (v) = X 2k (u A v) + fcAfc-iffcAfc-i - 2X k+l )uv 
and if k is even, then 

EB<® (u)B<® (v) = (X 2k - X 2 k ){u f\v) + kX k ({l — k/A)X k + kX k A 4 /4 - X k+2 )uv. 
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Remark 1.7. Based on the facts that A = 1, Ai = and A 2 = 1, (1-16) 
becomes EB^(u)B^\v) = uAv — uv for any < u, v < 1. That is, {B^(u), 
< u < 1} is a Brownian bridge. Hence, the result of Corollary 1.2 for /c = 1 
matches that in Remark 1.5. For k = 2, simple calculations from (1.16) show 
that EB^(u)B^ 2 \v) = (A 4 - l)(u A v - uv) for any < u,v < 1. Notice 
that V2 = ^4 — = ^1(^4 ~~ !)■ Thus, when k = 2, the result of Corol- 
lary 1.2 matches that of Corollary 1.1. In general, the Gaussian process 
{B^ k \u),0 < u < 1} for k > 3 depends on the moments of the innovation 
distribution and cannot be identified with a specific classic process such as 
a Brownian motion or Brownian bridge. 

Notice that Corollary 1.2 gives the weak convergence of the self- normalized 
high moment centered partial sum process {T^ (u)/a k n ^ , < u < 1} of resid- 
uals for a fixed k. The following result considers the joint weak convergence 
of two self-normalized high moment centered partial sum processes. 

Theorem 1.5. Assume that (1.5), (1.6), (1.13) and (1.14) hold. Assume 
also that k > 1 is an odd number and [13 = [i^ = fik+2 = l^2k+i = 0. Then 
E\£o\ 2( - k+1 ^ < 00 implies that 

converges weakly, in the Skorokhod space D 2 [0, 1] equipped with the product 
Ji topology, to a two-dimensional Gaussian process {(B^ k \u), B^ k+1 \v)), < 
u,v < 1}, where {B^ k \u),0 < u < 1} and {B^ k+1 \v), < v < 1} are two in- 
dependent Gaussian processes defined by (1.16). 

Remark 1.8. The conditions /13 = [i^ = fik+2 = ^2k+i = can be re- 
placed with the stronger condition that the innovation distribution is sym- 
metric about 0. 

2. Applications. This section considers applications of the high moment 
residual partial sums to a change-point problem, goodness-of-fit and the 
construction of a kernel density estimate of the unobservable innovation 
distribution. 

2.1. CUSUM tests for structural change of GARCH models. In this sub- 
section we consider the CUSUM normalized high moment partial sum pro- 
cess {Sn\u) — uSn(l),0 < u < 1} defined in Theorem 1.2. It is related 
to the standard CUSUM test introduced by Brown, Durbin and Evans [9], 
which was one of the first tests on structural change with unknown break 
point. 
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We first consider a structural change in the conditional mean for GARCH 
models. We can formulate it as the following hypothesis test. The null hy- 
pothesis is "no-change in the conditional mean," 



X t = a t e t , 

(2- 1 ) H °--\a? = a + f:a i Xl i + j:(3 j 

i=i j=i 



i = 0,l,...,n, 



and the alternative is "one change in the conditional mean," 
X t = o t e u 



H a :{ 



°t = "0 + J2 a i X t-i + Pj a t-j 
1=1 j=l 

X t = a t e t + /u, 

v i 
a 2 = a + J2 oii(Xt-i - n) 2 + Pj a t-j 
i=i j=i 



t = 0,..., [nu*], 



> , t= [nu*] + 1, ... ,n, 



where and < u* < 1 . 

To test the above hypothesis, we use the standard CUSUM test con- 
structed from residuals as 



max 

l<i<n &(n)V n 

By a straightforward calculation, it is easy to verify that 

CUSUMU= sup l ^ 1)( ^ ) -f" )(1)l +0 p(l), 



0<u<l ff(n)V n 

provided that Ee\ < oo. Therefore, by Corollary 1.1, under Hq, 

CUSUM w sup \Bq(u)\, 

0<u<1 

where {Bq(u),0 < u < 1} is a Brownian bridge. Hence, we can reject the Hq 
in favor of H a if CUSUM^ is large. 

Remark 2.1. The statistic CUSUM^ involves the estimator of 
y/jl-z- However, according to the GARCH model setup, \i2 = 1- Thus, the 
term oi n \ can be dropped in CUSUM^ to obtain a related test statistic. 



A second interesting structural change hypothesis concerns a change in 
the conditional variance of a GARCH model. We use the above Hq as the 
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null hypothesis for "no-change in the conditional variance" against the "one 
change in the conditional variance" alternative 



( X t = a t e t , 



where 



a + Y, a i x t~i + £ PjO-t-j > if t = 0, . . . , [ 



nu 



i=l 

p 



a 



3=1 



71, 



i=l 



(a 0) ai, • ■ • , at p ,{h, ■ ■ ■ , Pq) ^ («o, • • • , oc , (3[, . . . , ) 



and 



0<u* < 1. 

In the following we propose two CUSUM statistics. The first statistic is 
defined as 



CUSUM[ 2) = max ■ 

l<i<n 



where 



Pi 



i n 



n 



t=l 

is an estimator of v 2 = E( £ o ~ M2) 2 ; - The statistic O2 uses the fact that A2 = 1 
is known from the definition of the GARCH process and is not estimated; 
see (1.4). The second statistic is defined as 



CUSUM^ = max ■ 

z l<i<n 

r( 2 ) ; c 



(n)l 



that is, CUSUM 2 is centered about the residual sample mean e in contrast 

(2) 

to the no centering CUSUM\ . Again, by straightforward calculations, it is 
easy to show that 



CUSUM? = sup l^'M-^'W'+oHi) 

0<u<l 



?(2)/ 



and 



CUSUM { 2 2) = sup °" (n) 



0<«<1 ^2V n 



ri 2) (n) 



(7 



(«) 



op(1), 
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Table 1 





Size and power 


of CUSUM™ 


statistic for 


GARCH(1, 1) 




£o 


= N(0,1) 


n = 500 


n = 1000 


n = 1500 


n = 3000 





= (0.0002,0.1,0.7) (Null) 


0.0230 


0.0358 


0.0394 


0.0412 


0' 


= (0.0003,0.1,0.7) 


0.2342 


0.6484 


0.8752 


0.9958 


0' 


= (0.0002,0.167,0.7) 


0.1316 


0.3908 


0.5998 


0.9186 


0' 


= (0.0002,0.1,0.767) 


0.1792 


0.5470 


0.8264 


0.9924 





= (0.0002,0.1,0.8) (Null) 


0.0162 


0.0320 


0.0370 


0.0386 


0' 


= (0.0003,0.1,0.8) 


0.1040 


0.3922 


0.6570 


0.9642 


0' 


= (0.0002,0.167,0.8) 


0.1840 


0.6260 


0.8786 


0.9978 


0' 


= (0.0002,0.1,0.867) 


0.1768 


0.5980 


0.9320 


1.0000 



provided that Eeq < oo. Therefore, by Corollaries 1.1 and 1.2 (cf. Remark 1.7), 
under Hq, 

CUSUM\ 2) sup \B (u)\, i = l,2, 

0<«<1 

where {Bq(u),0 < u < 1} is a Brownian bridge. Hence, we can reject the Hq 

(2) 

in favor of H a i whenever CUSUM\ ' (i = 1,2) is large. 

Preliminary empirical studies show promising results from the above pro- 
posed test statistics. They outperform the CUSUM test constructed from 
the squares of the original data by Kim, Cho and Lee [17]. Some details are 
given in [23]. Independently, Kokoszka and Leipus [18] also study a change 
point for an ARCH process, again based on the original observations and 

(2) 

not residuals. Here we list empirical sizes and powers of the CUSUM%' test. 
The significance level is 5% with 1.358 as the critical value, the break point 
u* at the alternative is 0.5, and the number of replicates is 5000. Tables 1 

(2) 

and 2 show that there are size distortions of the CUSUM 2 test, but less 
serious with large sample size. The null and alternatives given in these ta- 
bles are slightly different from the cases studied in [17]. Their tables use ao 
as 0.2 or 0.3. When fitting GARCH models to financial stock returns data, 
typically a much smaller value of Qo is found and, hence, our tables use 
values of ao such as 0.0002 and 0.0003. A simulation with ao as 0.2 or 0.3 
was also undertaken, but not reported here. In addition, we include the near 
integrated GARCH cases with at = 0.1 or 0.167 and fa = 0.8 or 0.867 that 
were shown to have poor performance in Kim, Cho and Lee's test [17]. The 

(2) 

CUSUM 2 test outperforms Kim, Cho and Lee's test in all instances, with 
both large and small values of ao and, in particular, it has substantial power 
gains when the innovation distribution is t(8). 
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2.2. Jarque-Bera normality test. In this subsection we consider the self- 
normalized high moment centered partial sum process (1.15) for k = 3 and 
k = 4. They correspond to the sample skewness partial sum process as 

A f£\u)/n 

ln(u) = rrj , 0<1i<l, 

and the sample kurtosis process as 

«n(tl) = .4 ;/ , 0<U<1. 
CT (n) 

The sample skewness and kurtosis of the residuals are 7n(l) and K n (l), 
respectively. Omnibus statistics based on sample skewness and kurtosis have 
been used to test normality. Bowman and Shenton [8] and Gasser [13] give 
details of this method. The basic idea is to construct the statistic 

4(7 n (l)-A 3 ) 2 + 4(^(l)-A 4 ) 2 , 

where, by (1.16), 

= E(B® (l)) 2 = (A 6 - A|) + 3(3 + 3A| - 2A 4 ) + 3A 3 (A 3 /4 + 3A 3 A 4 /4 - A 5 ) 
and 

a 2 K = E(B^(1)) 2 = (A 8 - Al) + 4A 3 (4A 3 + 4A 3 A 4 - 2A 5 ) + 4A 4 (A^ - A 6 ). 

Assume that the innovation distribution is symmetric about 0. Then by 
Theorem 1.5 

(2-2) 4(7n(l) - A 3 ) 2 + 4(M1) " A 4 ) 2 X 2 2) . 

Table 2 





Size and Power 


of CUSUM™ 


statistic for 


GARCH (1, 1) 




£o 


= t(8) 


n = 500 


n = 1000 


n = 1500 


n = 3000 




= (0.0002,0.1,0.7) (Null) 


0.0234 


0.0302 


0.0336 


0.0396 


0' 


= (0.0003,0.1,0.7) 


0.1524 


0.4286 


0.6708 


0.9540 


0' 


= (0.0002,0.167,0.7) 


0.0752 


0.1986 


0.3234 


0.6620 


0' 


= (0.0002,0.1,0.767) 


0.1056 


0.3370 


0.5542 


0.9132 




= (0.0002,0.1,0.8) (Null) 


0.0188 


0.0256 


0.0318 


0.0378 


0' 


= (0.0003,0.1,0.8) 


0.0716 


0.2432 


0.4354 


0.8126 


0' 


= (0.0002,0.167,0.8) 


0.0932 


0.3450 


0.5758 


0.9308 


0' 


= (0.0002,0.1,0.867) 


0.1126 


0.3770 


0.7198 


0.9924 
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In the special case where the innovation distribution is standard normal, for 
which A3 = 0, A4 = 3, o -2 , = 6 and a\ = 24, then (2.2) becomes the Jarque- 



The statistic JB in (2.3) is the Jarque-Bera normality test widely used 
in econometrics and implemented in standard statistical packages such as 
S-PLUS, and Jarque and Bera [15] show that JB is a Lagrange multiplier 
test statistic of normality against alternatives within the Pearson family of 
distributions, which includes the beta, gamma and Student's t distributions 
among others. They point out that it is asymptotically equivalent to the 
likelihood ratio test, implying it has the same asymptotic power characteris- 
tics and, hence, has maximum local asymptotic power [11]. Therefore, a test 
based on JB is asymptotically locally most powerful against the Pearson 
family, and (2.3) shows that JB is asymptotically distributed as x 2 (2)- The 
hypothesis of normality is rejected for large sample size, if the computed 
value of JB is greater than the appropriate critical value of a X(2)- ^ u [19] 
has used Monte Carlo simulation to obtain critical values for several dif- 
ferent n. Based on this, a finite sample size correction can also be used to 
improve the choice of the critical value. Lu [19] obtained the finite sample 
size correction for the size 0.05 critical values of the JB test as 

JB om = 5.991645 - 15.17?i~ 1/2 + 345.971" 1 - 3110.8rr 3/2 , n > 100. 

We are unaware of any other results studying the Jarque-Bera test for 
GARCH residuals. Kilian and Demiroglu [16] studied the Jarque-Bera test 
for autoregressive residuals. 

2.3. Kernel density estimation of the innovation distribution. The om- 
nibus type statistic discussed in Section 2.2 provides a means to test a spe- 
cific type of unobservable innovation distribution, such as normal, Student-i 
and two-sided exponential. However, in practice, the normal innovation as- 
sumption is often rejected, as are other known types of distributions. Rather 
than focusing on identifying the innovation distribution to a specific mem- 
ber of a family, in this subsection we turn to a nonparametric kernel density 
estimation based on the residuals. This would be needed if one wished to 
implement a semi-parametric bootstrap methodology in this setting. 

Assume that the innovation distribution has a uniformly continuous den- 
sity function f(x) which is unknown. Let h n be a sequence of positive num- 
bers and K be a probability density function (kernel) with mean and vari- 
ance 1. Then the kernel density estimation of f(x) based on the residuals is 
defined as 
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Its counterpart based on i.i.d. innovations is denned as 

/»(*) = -^-E^f^). xel 

nh n ^ V h n J 



Theorem 2.1. Assume that (1.5), (1.6) and (1.14) ZioW. in addition, 
we assume that the following three conditions hold: 

(i) /i n > 0, /t n -* 0, v^^n o°; 

(ii) sup| x | >b \x\K(x)^0 as b^oo; 

(iii) K is Lipschitz, that is, there exists a constant C such that 

\K{x)-K{y)\<C\x-y\ Vx,y£lR. 
Then E\eq\ < oo implies that 

SUp\ f n (x) - f n (x)\ = Op(l). 

xeM. 

The proof of Theorem 2.2 follows easily from Theorem 1.3. Given the 
conditions in Theorem 2.2, we have (cf. [21]) 

sup|/ n (x) - f(x)\ =o P (l). 

Thus, by Theorem 2.2, 

sup|/ n (x) - f(x)\ =o P (l). 

Notice in the above result that only the finite first innovation moment and 
/n consistency of the parameter estimate are required. 



3. Proofs. This section begins with a proof of Theorem 1.1. It is given in 
a sketch or overview form, with the details given in a series of lemmas which 
are placed in the later part of this section. The proofs of Theorems 1.3 and 
1.4 rely on the proof of Theorem 1.1. 

Proof of Theorem 1.1. Let 

„ [nu] 

s t = ^, l<t<n and §£\u) = £e£, 0<«<1, 
°* t=i 

where dt is defined in (1.8). By (1.10) and (1.11), 



it = e t ( 1 + 
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and, hence, 



^* ) («)=sS fc) («)+E ; £ e ~ 



i=l 



t=l 



(7/ 



Thus, Theorem 1.1 follows if 



(3.1) sup 

0<«<1 

and 
(3.2) 



L(5W(u) - $<*>(«)) + Vn(S„ " »)) 



0P (1) 



t=i 



0"t - o-f 



0"i 



Op(1) 



1< i < k. 



The sample conditional standard deviation estimates at are uniformly 
bounded away from in probability. This is argued as follows. Since is 
-y/n-consistent, there exists an open ball in the interior of such that, for 
any small r] > 0, then 6 belongs to this open ball with probability > 1 — r] as 
n — ► oo. Therefore, a\ > olq > ^ao > with probability > 1 — 77. A stronger 
result than (3.2) is given in Lemma 3.5. 

Let 

^(aKe + n-^-am) 



(3.3) 

and 
(3.4) 



5t(u) 



l + n- 1 ^^) 



Though $t( u ) depends on n, we omit it for convenience of notation. 
Using (1.1), (1.7), (1.8), (3.3) and (3.4), we can rewrite it as 

e t = Et(y/n(9 n -B)), l<t<n, 

that is, it = £t(u) with u = ^/n{6 n — 6). 

Hence, by (1.5), (1.11) and (1.12), we can prove (3.1) if, for any b > 0, 



sup sup 

|u|<6 0<u<l 

This last part follows by 



L£ (e * (u )- e *) + ^ W 0), u 
m t=i z 



i n 

(3.5) sup —= 2_, 
|u|<6 V" t= i 

and 

(3.6) sup sup 
|u|<6 0<u<l 



e?(u)-ef 1 



2^ 



<01og<7?(0),u) 



E4(51oga t 2 (0),u)-u^(^),u) 



t=i 



op(l). 



op(1) 



op(l). 
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The proof of (3.6) follows by taking sup| u | <b into the inner product first, 
then applying Lemma 3.6 and noting that (Slogof (0),u) = h(e t -i,e t -2, ■ ■ •) 
for an appropriate function h. We also use the fact that Et and <91og<7 2 (6) 
are independent, and that, by Lemma 3.1, E(\d log a^(G)\) < oo. 

The main idea to prove (3.5) is to have a proper approximation of 

l/yjl + n~ 1 / 2 g t (u) so that 
1 



l + n-V^ii) 



1 



9tH | ( 1 
2y/n \y/n 

(dlo g( x 2 (fl),u) 
2y/n~ 



Op 



uniformly in |u| < b and 1 < t < n. Lemmas 3.1-3.4 are devoted to showing 
that this approximation holds. 

We divide the proof of (3.5) into two parts. By (3.4), equation (3.5) will 
follow if 



1 

su p ~^E N 

|u|<6 y/n t=1 



1 



l + n-V2^( u ) 



1 



(<91ogof(0),U 



2y/n~ 



o P (l) 



and 



(3.7) 



sup -pY] \e t \ 
|u|<6 V" t= i 



1 



(dlogo- 2 (0),u 
/fc(dlogcr 2 (0),u 



op(l). 



These are proven in Lemma 3.7. This completes the proof of Theorem 1.1. 

□ 



As a consequence of Theorem 1.1, we can obtain the consistency result 

Afe = 2n (l)/ra — > for k>2, where Tjl^ is given in (1.15). First, for any 
1 < i < k, Theorem 1.1 implies 

1 



-i ll -i It 

(3-8) -E^—E^-^ 



(1>(0),yfti(0 n -0))+Op 



t=l t=l 

In particular, since /ii = 0, we obtain 



(3.9) 



£ = Op 



Since n is y/n consistent, then for k > 2 and -E|eo| fc < °o, equation (3i 
implies that 



(3.10) 



fi fc) (i) ri fc) (i) 



71 



Op 



1 
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Proof of Theorem 1.3. Equation (3.5) implies that, for any b > 
and an integer k > 1, 



(3.11) 



sup 

M<&t=i 



n In 



o P (l). 



By the inequality ||a| — < \a — b\, (3.11) yields 



sup 
|u|<6 



1 n 
In j—i 



Si- 



2n 



t=i t=i 
Using ergodicity, we have, for each u G 



o P (l). 



1 



In Remark 1.3 it is argued that V'fc(u) is Lipschitz. With this method we 
obtain that (u) is also Lipschitz a.s. uniformly in n. Hence, one can 
obtain 



sup 1^ ; (u) - ^fc(u)| = Op(l). 
|u|<6 



Thus, 



1 n k 

^|4(u)-4|-^ fe (u) = 0P (1). 

t=l 2 



sup 
|u|<6 

Theorem 1.3 now follows standard arguments. This completes the proof of 
Theorem 1.3. □ 



Proof of Theorem 1.4. When k = l, we have 



(n) 



<7 



('0 



< l_ |fW(^)-T^(u)| |n 



.(1)/ 



,(1), 



7? 



tr 



7? 



1 



1 



CT (n) CT (n) 



Since -Beg < oo, the invariance principle for i.i.d. partial sums and (3.10) 
imply that 



sup 

0<m<1 



\tP(u) 



1 



1 



(n) *(n) 



op(1). 



Hence, we can prove Theorem 1.4 for the case k = 1 if 



sup 

0<u<l 



7? 



op(l), 
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which follows immediately by 

sup |f«(u)-rW(«)| 

0<«<1 



< sup \(s£\u)-uS£\l))-(S£\u)-uSV(l))\ + \i-e\ 

0<u<l 



and by Theorem 1.2 and (3.9). 

Next we consider the case k > 2. Let 



and 



/ -, \ nu \ \ 
L n (u) = ^L-Yj^Xu) - u\ k a\ n) \ 

/ -, M \ 

Ln (u) = V^" £T«(u) ~ ^<)J • 



Then 



sup —= 

0<«<1 v n 



fi k \ u ) rr» 



(*)/ 



a 



(n) 



a 



(n) 



sup 0<u<1 |L n (u)-L n (u)| 

< ; h SUp |-LnW 



Notice that (3.10) implies 



1 1 



& (n) °f») 



0<u<l 



°'<tr>- 



Therefore, we can prove Theorem 1.4 if 



(3.12) 

and 
(3.13) 



sup \L n (u) - L n (u) \ = o P (l) 

0<u<l 



\ L n(u)\ , , 

sup 7 =—=Op(\). 



""■f I — 
0<m<1 V n 



By the facts that e = Op(l/y/n) and = ^2 + op(l), (3.13) holds if 



sup 

0<u<l 



n 



J2 £ t ~ 



f=i 



op(1), 



which is true by Lemma 3.6. 
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To prove (3.12), we need a finer representation of <t^ in terms of &( n y 
By (3.8) and (3.9) we obtain 

*fn) = *(n) " ^(m, V^(0n ~ 0)) + Op ( 



"n \v n / 
By a first-order Taylor approximation with remainder we obtain 



1 \\ fc / 2 



(3.14) =(af -J*y,(9),M0n-0))) +°P 



k/2 / 1 



ka 1 ?- 2 



= a\ n) - -^L-kMO), yfrfin ~ 0)) + op (j= 

On the other hand, by (3.8) and (3.9), and the facts that e = Op{l/ \/n) 
and e = Op(l/y/n), we obtain 



1 1 l nU \ T.J \ nU \ 



1 [nu] *■«», ~ hp [nu] 

= 1 ^ fc _ ^ _ 0)) _ £ ,-l + Gp(1) 

= - ^(m, M°n ~ 0)) + op(1) 

uniformly in < u < 1. Substituting the above expression and (3.14) into 
(3.12) yields 

L n {u) = -^Ti k \u) - ^%(0), M0 n - 0)) 
\/n 2 



- U\ k (yfajf n) - ^-(m, M0n - 0))) + OP(I) 

= L{u)+o P {l) 

uniformly in < u < 1. This concludes the proof of (3.12) and, hence, The- 
orem 1.4. □ 

The remainder of this section gives the various lemmas needed in the 
proofs above, plus the proof of Theorem 1.5. 

By the mean value theorem for the multivariate function of (u) there is a 
C satisfying |£ — 6\ < \u\/-*/n so that (3.3) yields 

, , ,,, I^(0| |&tf(C)l *f (0 

w( u )l = l u l — TTTTT = l u l 



*H0) 1 1 o-KO oftey 
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Also, by a second-order term Taylor expansion there exists £ satisfying |£ — 
9\ < luj/y'n so that 

g t (u) = <Slogo 2 (0),u) + —u-j^V, 

where u T is the transpose of the vector u, and 9 2 o 2 (u) is the matrix of the 
second-order partial derivatives of of (u) (the Hessian matrix). Therefore, 

snp |a(u) - <91ogo 2 (0),u>| < * sup ^ffif'lE 

Thus, to show that <?t(u) has a finite moment and can be approximated by 
(Slogof (0),u) in the neighborhood of |u| < b for some b > 0, we need the 
following two lemmas. 

Lemma 3.1. If (1.6) and (1.4) /ioW, £/ien £'|eo| <5 < oo for some 5 > 
implies that 

\do*(u) 



£?( snp|aioga t 2 (u)|) K = £j%up^^ 
Vue© / Vue© of(u) 



< oo 



and 

/ |9 2 of(u) n 

£ sup J n^T^ <oo 

Vue© o t 2 (u) 

/or any k* > 0. 



Proof. See the proof of Lemma 5.6 of [3]. □ 

Lemma 3.2. // (1.6) and (1.14) ZioZd, i/ien -Eleol 5 < oo /or some 5 > 
implies that, for any b > and k* > 0, i/iere exists an integer N such that 

n>N \\u\<b O 2 



sup E\ sup -j— ) <oo. 



Proof. By Lemma 3.1 of [3], when n is large enough (so that 9 + 
n _1 / 2 u G 0), we have 

< *(0 + n~ 1 / 2 u) < d f max ^' + " ^ Yc^g) < Cip^O), 

< i < oo, 

where Ci is a constant and 1 < p n = 1 + n^^b/u, where u is defined 
above (1.6). Thus, Lemma 3.2 will be proven if we can show that 

„/- E:EipW»)*L , 
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Since pjy can be close enough to 1 if N is large enough, the above inequality 
follows from the same proof of Lemma 3.7 of [2]. For completeness, we give 
a detailed proof here. 

By Lemma 3.1 of [3], there are constants C2 and < p < 1 such that 

|q(u)| < C2P 1 for all u G and all i. 

Then for any M > 1, we have 

t + 2^i=l c i{V)*t-i i=M+l 

00 

< P fJ + c 2 E fapfxli. 

i=M+l 

By Lemma 2.3 of [3], 

(3.15) there exists 5* > such that E'lXol 5 < 00. 

Notice that, for N sufficiently large, p^p < 1. By Markov's inequality, we 
have, for x > p 2 N , 

p{ E (pnpT xl^x/2) 

U=M+1 J 

00 

< E P{X t 2 _ i >(a ; /2)(p Jv p)-(l-(p A rp) 1 /2)(p Jv p)V2 } 

i=M+l 
00 

= E P{|^| 5 *X-/2) 5 * /2 (1-W) 1/2 ) 5 * /2 (pvp)^* /4 } 



i=M+l 

< E|X„f {x/2)S*/\l - (p NP f 2 r S ' /2 {l - {pNpfl A T\PNp) M5 * /A . 
Choosing M = log(C2x/2)/logpAr, we have, for any k* > 0, 

itts^w; 2 

<W E (PN P yxi, t >x/2\ 

U=M+1 ) 

< C 3 exp(-(5*/4)(l + logp^ 1 /logp7v)log(x/2)) 

if Pat > 1 is close enough to 1 (when iV is sufficiently large), and where C3, 
C4 are constants that may depend on N. □ 
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Using Lemmas 3.1 and 3.2 and Holder's inequality, we immediate arrive 
at the following result. 

Lemma 3.3. If (1.6) and (1.14) hold, then E\eo\ 5 < oo for some 5 > 
implies that, for any b > and k* > 0, there exists an integer N such that 



sup Ei sup |<?t(u)| I < oo 

n>N \|u|<6 / 



n>N \|u| 

and 

sup E[ sup y/n\g t (u) - (<91ogcr t 2 (0), u)| ) < oo. 

n>N V|u|<6 / 

Lemma 3.4. Suppose that (1.6) and (1.14) hold and E\£q\ & < oo for 
some 5 > 0. T/ien /or any 6 > 0, 



max sup 

l<t<«| u |<fe 



f i <01og^(0),u) 



l + ri-V2 fft ( u ) V 2 v^ 



Op 



Proof. First we state a well-known result that if {Y n ,n > 0} is a se- 
quence of identically distributed r.v.'s with £?|Yo| K < 00 f° r some At* > 0, 
then 

(3.16) max \Y t \ = o P (n 1/K *). 

l<t<n 

Thus, by Lemma 3.3 for k* > 2, and noting that by construction <7t(u) has 
the same marginal distribution for each t, we have 

n~ 1/2 max sup |fft(u)| = o P (n l/ti *~ l/2 ). 

1<*<"|u|<6 

This, together with the inequality that |l/\/l + x — 1 + x/2\ < 3x 2 for |x| < 
1/2, implies that 



max sup 

!<*<". | u |<6 



i = ~ f 1 " = ( P(- 1/K *~ 1/2 )) 2 = oMn 2 /^ 1 ). 



By Lemma 3.3 and (3.16), 



max sup y/n\gt(u) — (cHogo" 2 (0),u)| =op(n 1//fc *). 

l<t<n\ u \< b 

Choosing k* > 4 completes the proof of Lemma 3.4. □ 
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Lemma 3.5. Suppose that (1.6) and (1.14) hold and that E(\e \ 5 ) < oo 
for some 5 > 0. Then for any integers k,£> 1, 

n 

J2\et\ k \<rt-ttf = Op(l). 

t=i 

Proof. By (1.6) and (1.5), for any small rj > there exist a set with 
probability > 1 — rj and a constant C5 such that 

at > C5 and at > C5 for all t > 1. 

Thus, by (1.7), (1.8) and (1.9), there is a constant C§ such that 



N>t-<7 t r<(2C 5 )-*M fc sup 

wee 



< 



By Lemma 5.1 of [3], taking < v = <5/4 < 5/2 in their lemma 

/ <r?(0)\ 6/4 

£ sup ' ; < 00. 



2 

t-i 



By (3.15), and taking 5*/2<l, 



E 



E 

,i=i+l 



S*/2- 



< e P i5 * /2 ^ 



t-i 



i=t+l 



<E\X 



6* P 



t8*/2 



l-p S */ 2 ' 



Hence, by Holder's inequality 

(n \ s " I oo \ $** 

E %\ h \*t - &t\ l \ < E[ E folVt - < 00 

for sufficiently small 5** > 0. Thus, Lemma 3.5 is now proven. □ 

Lemma 3.6. Let Yt = h(et,£t-i, ■ ■ •) and suppose that E\Yq\ < 00. Then 



sup 

0<u<l 



-J2 Y t-uEY 
n f— f 



t=l 



o P (l). 
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Proof. For any < £ < 1 and for large n, 



sup 

0<M<1 



1 



J? 



Y,Y t -uEY Q 



t=i 



1 



<-EN + ^i y oi 



t=l 



+ sup 

£<u<l 



■;?■ 



< 



E(N-^oi) 



t=i 



+ 3^|y | + sup 

3>H] 



t=i 



Since Yj = h(et,£t-i, ■ ■ •), by Theorem 3.5.8 of [22] {Yj} is stationary and 
ergodic. Thus, as n — ► oo, 



1 



K] 



RJ E(I^|-^o|) = op(1) 



and 



sup 

j>K] 



-Em-^o: 



op(1). 



□ 



Lemma 3.7. Suppose that (1.6) and (1.14) /ioZd. Then, for any b > 
and on integer k>l, E\eo\ k < oo implies: 



(i) sup EM' 

M<&i=i 



1 



and 



1 n 

(ii) SU p ^| et |fc 



t=l 



(d\o g o-?(e),u) 

2y^ 



l + n-V2^( u ) 



1 



fc(dlogcx t 2 (fl),u) 
2v^ 



(<91o gC r 2 (fl),u) 

2^ 



P (1) 



0P (1). 



Proof. Part (i), case k = 1 is trivial. Part (ii), case = 1 follows directly 
from Lemma 3.4. 

It is easy to see from Lemma 3.1 that 

E sup |(dlog(7 t 2 (0),u)| fc < oo. 
|u|<6 

Now consider k>2. Thus, using the fact that St and d log erf (0) are in- 
dependent, we have, for 2 < i < k, 



i^su P -f>/|<dlog^(0),u) 

\lul<6 n i=l 
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1 n 

<-Y,E\e t \ k E S up\(dlogaUO),vL)\ l 
|»|<6 



t=i 
= 0(1). 

This, together with the binomial formula, implies Lemma 3.7(i). 
Now consider part (ii) and k>2. By Lemma 3.4 we have 



(aio g ^(g),u) 

1 ; — ^= h Op 



1 



'l + n-i/2^( u ) 2 v^ 
uniformly in |u| < b and 1 < t < n. Thus, using the inequality 

\{x + A) k -x k \ < k2 k - 1 \A\(\x\ k - 1 + lA]^ 1 ), 

we have 



1 



l + n-V2^(u) 



In 



1 



(dlogQ 2 (fl),u) 



<dlogo- 2 (0),u) 



2^ 



fc-i 



+ Op 



uniformly in |u| < b and 1 < t < n. Finally, using the fact that Sf and 
<91ogo" 2 (#) are independent and E sup\ u \ <b \(d\ogo~t(0),u)\ k < oo, the proof 
of Lemma 3.7(h) is immediate. □ 

Lemma 3.8. If EsQ k < oo for an integer k>2, then 



1 (Tl?\u) 



nuXu ) , < u < 1 



(n) 

converges weakly to the Gaussian process {B^ k \u),0 < u < 1} wrai/i covari- 
ance defined by (1.16). 

Proof. Using the standard GARCH scaling assumption ^2 = 1, we 
have Afc = fi^- Similar to the proof of Lemma 3.7, by Lemma 3.6 and e = 
Op(l/y/n) we have 



1 



1 



k 



t=i 



t=i 



t=i 



1 



j [mi] , ^ 

_ E( e < - Pk)- uk^ k _ x e + ufi k + o P [—= 
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uniformly in < u < 1. On the other hand, by e 2 = Op (1/n) we have 

\ k 1 2 / -t n \ ^ / 2 




e)2 = I 1 + - > fe? - 1) I +o P 



i + Af^-!)^) 



Therefore, 



Ti fc) (n) 




Wfc) " ^rJ2M£t ~ 1) + 2/i fe _ l£t ) ) +o P (l) 



2 *=i 



4-AfW(u) + . 



.„ n ^, , Op(l) 

<) 

uniformly in < u < 1. Since <r^ x — > 1 in probability, we can prove Lemma 3.8 

if{Mi k \u),0<u< 1} converges weakly to the Gaussian process {-B^(u), < 
u < 1}. According to the invariance principle for i.i.d. partial sums, 
{n -i/2 E M( £ fe_ Mfc))0 < u <l}and {n- 1 /2 uEr=i(/ife(e 2_ 1) + 2/Ufc _ ie<))0 < 
u < 1} are tight (in fact, they converge weakly to Gaussian processes) and, 
hence, so is the process {M^\u), < u < 1}. Using the form of Mn\u) and 
relatively easy but lengthy covariance computations, we obtain (1.16). These 
straightforward details are omitted. This completes the proof of Lemma 3.8. 
□ 

PROOF of Theorem 1.5. Using the form of Mn\u) from the proof of 
Lemma 3.8, we just need to show that, for any < u,v < 1, as n — > oo, 

EM^ k \u)M^ +1 \v) ^ 0. 

This computation is straightforward but lengthy. The details are omitted. 
□ 
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